Differences-in-Difference:

Difference-in-Difference:

Extend the interrupted time-series:

• Compare \(post - pre\) change in cases with “treatment” units versus “control”
  • \(Treated_{post} - Treated_{pre}\) (first difference)
  • \(Control_{post} - Control_{pre}\) (first difference)
  • \((Treated_{post} - Treated_{pre}) - (Control_{post} - Control_{pre})\) (Second difference)
• This can reveal the unbiased causal effect, assuming “parallel trends”
  • That is to say: “control” cases are not counterfactuals for “treated” case, but are the counterfactual trend

Difference-in-Difference:

Difference-in-Difference:

If parallel trends assumption holds, what kinds of confounding does this design eliminate?

• Any time-invariant confounders within units
• Any time-varying confounders that are shared by both treated cases and un-treated cases

• Crucial to choose right “control” cases to have parallel trends or e.g. share as many time-varying confounders.

Difference-in-Difference:

Implementation:

• With two “treatment” conditions, two time periods: calculate differences in means
• Use regression, with interaction term.

Difference-in-Difference:

Example: Card and Krueger (2000)

Do increases in the minimum wage increase unemployment in fast food?

• Examine bordering areas in New Jersey and Pennsylvania before and after increase in NJ minimum wage

Difference-in-Difference:

Two ways to use regression:

\[Y_it = \beta_0 + \beta_1 treat_i + \beta_2 post_{t} + \beta_3 treat_i \times post_{t} + \epsilon_{it}\]

Where \(treat_i\) is an indicator for being a unit that is ever treated. \(post_t\) is an indicator for the observation being after the treatment takes place (1 if yes, 0 if no).

\[Y_{post} - Y_{pre} = \beta_0 + \beta_1 treat_i + \epsilon_i\]

Difference-in-Difference:

How do we validate the parallel trends assumption?

• We cannot test fully, but we can check.

Placebo Tests:

• If “treatment” happens between time \(t\) and \(t+1\), we should not see an effect between \(t-1\) and \(t\)?
• If design is right, there should be no effects of the treatment prior to the treatment taking place.
• If we “pass” placebo test, does not reject possibility that something other than “treatment” caused a change.
• You need multiple pre-treatment data points

Difference-in-Difference:

Difference-in-Difference:

If we have multiple time-periods, multiple cases, we get generalized difference-in-differences:

• case-specific intercepts (dummy for each case: e.g. “country fixed effects”): capture time-invariant confounders within cases
• time-specific intercepts (dummy for each period of time): capture time-varying confounders that are shared across cases

Difference-in-Difference:

\[Y_{it} = \alpha_i + \alpha_t + \beta X_{it} + \epsilon_{it}\]

• \(\alpha_i\) is a dummy for each unit \(i\)
• \(\alpha_t\) is a dummy for each time period \(t\)
• \(X_{it}\) is an indicator for the presence of the treatment in case \(i\) and time \(t\)
• Why does the interaction of treatment and time period drop out of the model?

Difference-in-Difference:

This model is equivalent to the following:

\[Y_{it} - \overline{Y_i} = (\beta X_{it} - \overline{X_i}) + (\epsilon_{it} - \overline{\epsilon_i})\]

Or taking the difference of \(X\) and \(Y\) of each case from its mean. Any variables \(Z_i\) that do not vary over time are removed (and thus cannot confound).

Fixed Effects

Example:

Fixed Effects

Example:

Fixed Effects

Example:

summary(lm(y ~ x, df))$coefficients

##              Estimate Std. Error  t value    Pr(>|t|)
## (Intercept) 2.4999996 0.70803943 3.530876 0.002387267
## x           0.1949204 0.05910599 3.297812 0.004000581

Fixed Effects

Example:

Fixed Effects

In this case, the overall relationship between \(X\) and \(Y\) is positive. But within each unit (\(a \dots e\)), the relationship is negative!

Unobserved factors might make each unit have higher levels of x and higher levels of y, but as x increases within a unit (where these unobserved factors are constant), y decreases.

fixed effects allow us to extract this within unit relationship between \(X\) and \(Y\)

Contrast to pooled effects where we compare all observations to each other, not accounting for any unit-specific effects

Fixed Effects

How to use fixed effects

We add dummy variables for each unit

What does this do?

• Estimates a separate intercept for each unit (or group) within the data
• Residual of \(X\), is now the variation in \(X\) around the mean within that unit.
• Eliminates any confounding related to unchanging attributes of the unit

• Why is that?

• Doesn’t account for any confounding by variables that change over time

Fixed Effects

Fixed effects results:

summary(lm(y ~ x + g, df))$coefficients

##              Estimate Std. Error   t value     Pr(>|t|)
## (Intercept)  5.259039 0.13648714  38.53138 1.305777e-15
## x           -1.064946 0.03860439 -27.58612 1.325545e-13
## gb           5.183240 0.20609107  25.15024 4.721027e-13
## gc          10.568614 0.33765055  31.30045 2.322069e-14
## gd          15.645418 0.48294082  32.39614 1.443275e-14
## ge          20.950502 0.63257043  33.11964 1.063356e-14

Fixed Effects

An Example:

What was the effect of enlistment in the US Civil War on voting for the Republican party?

\[GOP_{ie} = \alpha_i + \alpha_e + \beta Enlist_i \times PostWar_e + \epsilon_y + \epsilon_i\]

• \(i\) is a county
• \(e\) is a state-election (election year in a specific state)
• \(y\) is a year between 1854 and 1880
• \(PostWar\) indicates if election is after 1861 (\(1\)) or before (\(0\))

An Example:

dd1 = felm(gop_vs ~ postbellum:vet_alt_pct  |  #interaction of enlistment rate, post-war
                    target_fips + state_election  |  #county, state-election fixed effects
                    0 | #no Instrumental variable
                    target_fips + election_year , # cluster errors by county, election year
           merged_use) 

An Example:

A placebo test/relaxed assumptions

\[GOP_{ie} = \alpha_{i} + \alpha_{e} + \sum_{y = 1854}^{1920} \beta_y EnlistmentRate_i * Year_y + \epsilon_i + \epsilon_y\]

An Example:

year_levels = c(1860, seq(1854,1858,2), seq(1862,1920,2)) %>% as.character()
merged_panel[, year_f := factor(election_year, levels = year_levels)]

g1 = felm(gop_vs ~ 1 + year_f*vet_alt_pct | #interaction of YEAR and enlistment-rate
                  target_fips + state_election | #county and state-election FE
                  0 | #no IV
                  target_fips + election_year ,#cluster by county/ election year
                  merged_use)

An Example:

Differences-in-Differences:

Key Assumptions:

• parallel trends: treated and untreated have same changes over time except for shift in treatment.
• equivalently: no time-varying omitted variable bias; no omitted variables correlated with changes in treatment, changes in outcome

Caveats:

• careful: check parallel trends
• careful: what happens when units are treated at different times?
• careful: STANDARD ERRORS! Clustering may not save you with small \(N\).

Natural Experiments:

Address confounding / omitted variable bias in a different way:

• exploit random or as-if random allocation of \(X\) to find causal effect
• look for “naturally” occurring randomization

Natural Experiments:

Distinguishing “natural experiment” from experiments:

• Treatment and control groups
• Random assignment (maybe)
• “as-if” random assigment
• no manipulation

Natural Experiments:

An observational study where causal inference comes from the design that draws on randomization.

• contrast to difference-in-difference: not a “conditioning” approach
• depends on argument to validate “as-if” random assumption, fewer assumptions in the statistical model

Natural Experiments:

Regression Discontinuity

• “Treatment” occurs for cases on one side of threshold, not to cases on other side
• Near the “threshold”, cases are arbitrarily assigned.
  • treatment can be for ALL cases on the “Treated” side (sharp RD)
  • treatment can increase at the threshold (“fuzzy” RD) (e.g. Mo 2018)

Natural Experiments

## List of 57
##  $ line                 :List of 6
##   ..$ colour       : chr "black"
##   ..$ size         : num 0.5
##   ..$ linetype     : num 1
##   ..$ lineend      : chr "butt"
##   ..$ arrow        : logi FALSE
##   ..$ inherit.blank: logi TRUE
##   ..- attr(*, "class")= chr [1:2] "element_line" "element"
##  $ rect                 :List of 5
##   ..$ fill         : chr "white"
##   ..$ colour       : chr "black"
##   ..$ size         : num 0.5
##   ..$ linetype     : num 1
##   ..$ inherit.blank: logi TRUE
##   ..- attr(*, "class")= chr [1:2] "element_rect" "element"
##  $ text                 :List of 11
##   ..$ family       : chr ""
##   ..$ face         : chr "plain"
##   ..$ colour       : chr "black"
##   ..$ size         : num 11
##   ..$ hjust        : num 0.5
##   ..$ vjust        : num 0.5
##   ..$ angle        : num 0
##   ..$ lineheight   : num 0.9
##   ..$ margin       :Classes 'margin', 'unit'  atomic [1:4] 0 0 0 0
##   .. .. ..- attr(*, "valid.unit")= int 8
##   .. .. ..- attr(*, "unit")= chr "pt"
##   ..$ debug        : logi FALSE
##   ..$ inherit.blank: logi TRUE
##   ..- attr(*, "class")= chr [1:2] "element_text" "element"
##  $ axis.title.x         :List of 11
##   ..$ family       : NULL
##   ..$ face         : NULL
##   ..$ colour       : NULL
##   ..$ size         : NULL
##   ..$ hjust        : NULL
##   ..$ vjust        : num 1
##   ..$ angle        : NULL
##   ..$ lineheight   : NULL
##   ..$ margin       :Classes 'margin', 'unit'  atomic [1:4] 5.5 0 0 0
##   .. .. ..- attr(*, "valid.unit")= int 8
##   .. .. ..- attr(*, "unit")= chr "pt"
##   ..$ debug        : NULL
##   ..$ inherit.blank: logi TRUE
##   ..- attr(*, "class")= chr [1:2] "element_text" "element"
##  $ axis.title.x.top     :List of 11
##   ..$ family       : NULL
##   ..$ face         : NULL
##   ..$ colour       : NULL
##   ..$ size         : NULL
##   ..$ hjust        : NULL
##   ..$ vjust        : num 0
##   ..$ angle        : NULL
##   ..$ lineheight   : NULL
##   ..$ margin       :Classes 'margin', 'unit'  atomic [1:4] 0 0 5.5 0
##   .. .. ..- attr(*, "valid.unit")= int 8
##   .. .. ..- attr(*, "unit")= chr "pt"
##   ..$ debug        : NULL
##   ..$ inherit.blank: logi TRUE
##   ..- attr(*, "class")= chr [1:2] "element_text" "element"
##  $ axis.title.y         :List of 11
##   ..$ family       : NULL
##   ..$ face         : NULL
##   ..$ colour       : NULL
##   ..$ size         : NULL
##   ..$ hjust        : NULL
##   ..$ vjust        : num 1
##   ..$ angle        : num 90
##   ..$ lineheight   : NULL
##   ..$ margin       :Classes 'margin', 'unit'  atomic [1:4] 0 5.5 0 0
##   .. .. ..- attr(*, "valid.unit")= int 8
##   .. .. ..- attr(*, "unit")= chr "pt"
##   ..$ debug        : NULL
##   ..$ inherit.blank: logi TRUE
##   ..- attr(*, "class")= chr [1:2] "element_text" "element"
##  $ axis.title.y.right   :List of 11
##   ..$ family       : NULL
##   ..$ face         : NULL
##   ..$ colour       : NULL
##   ..$ size         : NULL
##   ..$ hjust        : NULL
##   ..$ vjust        : num 0
##   ..$ angle        : num -90
##   ..$ lineheight   : NULL
##   ..$ margin       :Classes 'margin', 'unit'  atomic [1:4] 0 0 0 5.5
##   .. .. ..- attr(*, "valid.unit")= int 8
##   .. .. ..- attr(*, "unit")= chr "pt"
##   ..$ debug        : NULL
##   ..$ inherit.blank: logi TRUE
##   ..- attr(*, "class")= chr [1:2] "element_text" "element"
##  $ axis.text            :List of 11
##   ..$ family       : NULL
##   ..$ face         : NULL
##   ..$ colour       : chr "grey30"
##   ..$ size         :Class 'rel'  num 0.8
##   ..$ hjust        : NULL
##   ..$ vjust        : NULL
##   ..$ angle        : NULL
##   ..$ lineheight   : NULL
##   ..$ margin       : NULL
##   ..$ debug        : NULL
##   ..$ inherit.blank: logi TRUE
##   ..- attr(*, "class")= chr [1:2] "element_text" "element"
##  $ axis.text.x          :List of 11
##   ..$ family       : NULL
##   ..$ face         : NULL
##   ..$ colour       : NULL
##   ..$ size         : NULL
##   ..$ hjust        : NULL
##   ..$ vjust        : num 1
##   ..$ angle        : NULL
##   ..$ lineheight   : NULL
##   ..$ margin       :Classes 'margin', 'unit'  atomic [1:4] 2.2 0 0 0
##   .. .. ..- attr(*, "valid.unit")= int 8
##   .. .. ..- attr(*, "unit")= chr "pt"
##   ..$ debug        : NULL
##   ..$ inherit.blank: logi TRUE
##   ..- attr(*, "class")= chr [1:2] "element_text" "element"
##  $ axis.text.x.top      :List of 11
##   ..$ family       : NULL
##   ..$ face         : NULL
##   ..$ colour       : NULL
##   ..$ size         : NULL
##   ..$ hjust        : NULL
##   ..$ vjust        : num 0
##   ..$ angle        : NULL
##   ..$ lineheight   : NULL
##   ..$ margin       :Classes 'margin', 'unit'  atomic [1:4] 0 0 2.2 0
##   .. .. ..- attr(*, "valid.unit")= int 8
##   .. .. ..- attr(*, "unit")= chr "pt"
##   ..$ debug        : NULL
##   ..$ inherit.blank: logi TRUE
##   ..- attr(*, "class")= chr [1:2] "element_text" "element"
##  $ axis.text.y          :List of 11
##   ..$ family       : NULL
##   ..$ face         : NULL
##   ..$ colour       : NULL
##   ..$ size         : NULL
##   ..$ hjust        : num 1
##   ..$ vjust        : NULL
##   ..$ angle        : NULL
##   ..$ lineheight   : NULL
##   ..$ margin       :Classes 'margin', 'unit'  atomic [1:4] 0 2.2 0 0
##   .. .. ..- attr(*, "valid.unit")= int 8
##   .. .. ..- attr(*, "unit")= chr "pt"
##   ..$ debug        : NULL
##   ..$ inherit.blank: logi TRUE
##   ..- attr(*, "class")= chr [1:2] "element_text" "element"
##  $ axis.text.y.right    :List of 11
##   ..$ family       : NULL
##   ..$ face         : NULL
##   ..$ colour       : NULL
##   ..$ size         : NULL
##   ..$ hjust        : num 0
##   ..$ vjust        : NULL
##   ..$ angle        : NULL
##   ..$ lineheight   : NULL
##   ..$ margin       :Classes 'margin', 'unit'  atomic [1:4] 0 0 0 2.2
##   .. .. ..- attr(*, "valid.unit")= int 8
##   .. .. ..- attr(*, "unit")= chr "pt"
##   ..$ debug        : NULL
##   ..$ inherit.blank: logi TRUE
##   ..- attr(*, "class")= chr [1:2] "element_text" "element"
##  $ axis.ticks           :List of 6
##   ..$ colour       : chr "grey20"
##   ..$ size         : NULL
##   ..$ linetype     : NULL
##   ..$ lineend      : NULL
##   ..$ arrow        : logi FALSE
##   ..$ inherit.blank: logi TRUE
##   ..- attr(*, "class")= chr [1:2] "element_line" "element"
##  $ axis.ticks.length    :Class 'unit'  atomic [1:1] 2.75
##   .. ..- attr(*, "valid.unit")= int 8
##   .. ..- attr(*, "unit")= chr "pt"
##  $ axis.line            : list()
##   ..- attr(*, "class")= chr [1:2] "element_blank" "element"
##  $ axis.line.x          : NULL
##  $ axis.line.y          : NULL
##  $ legend.background    :List of 5
##   ..$ fill         : NULL
##   ..$ colour       : logi NA
##   ..$ size         : NULL
##   ..$ linetype     : NULL
##   ..$ inherit.blank: logi TRUE
##   ..- attr(*, "class")= chr [1:2] "element_rect" "element"
##  $ legend.margin        :Classes 'margin', 'unit'  atomic [1:4] 0.2 0.2 0.2 0.2
##   .. ..- attr(*, "valid.unit")= int 1
##   .. ..- attr(*, "unit")= chr "cm"
##  $ legend.spacing       :Class 'unit'  atomic [1:1] 0.4
##   .. ..- attr(*, "valid.unit")= int 1
##   .. ..- attr(*, "unit")= chr "cm"
##  $ legend.spacing.x     : NULL
##  $ legend.spacing.y     : NULL
##  $ legend.key           :List of 5
##   ..$ fill         : chr "white"
##   ..$ colour       : logi NA
##   ..$ size         : NULL
##   ..$ linetype     : NULL
##   ..$ inherit.blank: logi TRUE
##   ..- attr(*, "class")= chr [1:2] "element_rect" "element"
##  $ legend.key.size      :Class 'unit'  atomic [1:1] 1.2
##   .. ..- attr(*, "valid.unit")= int 3
##   .. ..- attr(*, "unit")= chr "lines"
##  $ legend.key.height    : NULL
##  $ legend.key.width     : NULL
##  $ legend.text          :List of 11
##   ..$ family       : NULL
##   ..$ face         : NULL
##   ..$ colour       : NULL
##   ..$ size         :Class 'rel'  num 0.8
##   ..$ hjust        : NULL
##   ..$ vjust        : NULL
##   ..$ angle        : NULL
##   ..$ lineheight   : NULL
##   ..$ margin       : NULL
##   ..$ debug        : NULL
##   ..$ inherit.blank: logi TRUE
##   ..- attr(*, "class")= chr [1:2] "element_text" "element"
##  $ legend.text.align    : NULL
##  $ legend.title         :List of 11
##   ..$ family       : NULL
##   ..$ face         : NULL
##   ..$ colour       : NULL
##   ..$ size         : NULL
##   ..$ hjust        : num 0
##   ..$ vjust        : NULL
##   ..$ angle        : NULL
##   ..$ lineheight   : NULL
##   ..$ margin       : NULL
##   ..$ debug        : NULL
##   ..$ inherit.blank: logi TRUE
##   ..- attr(*, "class")= chr [1:2] "element_text" "element"
##  $ legend.title.align   : NULL
##  $ legend.position      : chr "right"
##  $ legend.direction     : NULL
##  $ legend.justification : chr "center"
##  $ legend.box           : NULL
##  $ legend.box.margin    :Classes 'margin', 'unit'  atomic [1:4] 0 0 0 0
##   .. ..- attr(*, "valid.unit")= int 1
##   .. ..- attr(*, "unit")= chr "cm"
##  $ legend.box.background: list()
##   ..- attr(*, "class")= chr [1:2] "element_blank" "element"
##  $ legend.box.spacing   :Class 'unit'  atomic [1:1] 0.4
##   .. ..- attr(*, "valid.unit")= int 1
##   .. ..- attr(*, "unit")= chr "cm"
##  $ panel.background     :List of 5
##   ..$ fill         : chr "white"
##   ..$ colour       : logi NA
##   ..$ size         : NULL
##   ..$ linetype     : NULL
##   ..$ inherit.blank: logi TRUE
##   ..- attr(*, "class")= chr [1:2] "element_rect" "element"
##  $ panel.border         :List of 5
##   ..$ fill         : logi NA
##   ..$ colour       : chr "grey20"
##   ..$ size         : NULL
##   ..$ linetype     : NULL
##   ..$ inherit.blank: logi TRUE
##   ..- attr(*, "class")= chr [1:2] "element_rect" "element"
##  $ panel.spacing        :Class 'unit'  atomic [1:1] 5.5
##   .. ..- attr(*, "valid.unit")= int 8
##   .. ..- attr(*, "unit")= chr "pt"
##  $ panel.spacing.x      : NULL
##  $ panel.spacing.y      : NULL
##  $ panel.grid.major     :List of 6
##   ..$ colour       : chr "grey92"
##   ..$ size         : NULL
##   ..$ linetype     : NULL
##   ..$ lineend      : NULL
##   ..$ arrow        : logi FALSE
##   ..$ inherit.blank: logi TRUE
##   ..- attr(*, "class")= chr [1:2] "element_line" "element"
##  $ panel.grid.minor     :List of 6
##   ..$ colour       : chr "grey92"
##   ..$ size         : num 0.25
##   ..$ linetype     : NULL
##   ..$ lineend      : NULL
##   ..$ arrow        : logi FALSE
##   ..$ inherit.blank: logi TRUE
##   ..- attr(*, "class")= chr [1:2] "element_line" "element"
##  $ panel.ontop          : logi FALSE
##  $ plot.background      :List of 5
##   ..$ fill         : NULL
##   ..$ colour       : chr "white"
##   ..$ size         : NULL
##   ..$ linetype     : NULL
##   ..$ inherit.blank: logi TRUE
##   ..- attr(*, "class")= chr [1:2] "element_rect" "element"
##  $ plot.title           :List of 11
##   ..$ family       : NULL
##   ..$ face         : NULL
##   ..$ colour       : NULL
##   ..$ size         :Class 'rel'  num 1.2
##   ..$ hjust        : num 0
##   ..$ vjust        : num 1
##   ..$ angle        : NULL
##   ..$ lineheight   : NULL
##   ..$ margin       :Classes 'margin', 'unit'  atomic [1:4] 0 0 6.6 0
##   .. .. ..- attr(*, "valid.unit")= int 8
##   .. .. ..- attr(*, "unit")= chr "pt"
##   ..$ debug        : NULL
##   ..$ inherit.blank: logi TRUE
##   ..- attr(*, "class")= chr [1:2] "element_text" "element"
##  $ plot.subtitle        :List of 11
##   ..$ family       : NULL
##   ..$ face         : NULL
##   ..$ colour       : NULL
##   ..$ size         :Class 'rel'  num 0.9
##   ..$ hjust        : num 0
##   ..$ vjust        : num 1
##   ..$ angle        : NULL
##   ..$ lineheight   : NULL
##   ..$ margin       :Classes 'margin', 'unit'  atomic [1:4] 0 0 4.95 0
##   .. .. ..- attr(*, "valid.unit")= int 8
##   .. .. ..- attr(*, "unit")= chr "pt"
##   ..$ debug        : NULL
##   ..$ inherit.blank: logi TRUE
##   ..- attr(*, "class")= chr [1:2] "element_text" "element"
##  $ plot.caption         :List of 11
##   ..$ family       : NULL
##   ..$ face         : NULL
##   ..$ colour       : NULL
##   ..$ size         :Class 'rel'  num 0.9
##   ..$ hjust        : num 1
##   ..$ vjust        : num 1
##   ..$ angle        : NULL
##   ..$ lineheight   : NULL
##   ..$ margin       :Classes 'margin', 'unit'  atomic [1:4] 4.95 0 0 0
##   .. .. ..- attr(*, "valid.unit")= int 8
##   .. .. ..- attr(*, "unit")= chr "pt"
##   ..$ debug        : NULL
##   ..$ inherit.blank: logi TRUE
##   ..- attr(*, "class")= chr [1:2] "element_text" "element"
##  $ plot.margin          :Classes 'margin', 'unit'  atomic [1:4] 5.5 5.5 5.5 5.5
##   .. ..- attr(*, "valid.unit")= int 8
##   .. ..- attr(*, "unit")= chr "pt"
##  $ strip.background     :List of 5
##   ..$ fill         : chr "grey85"
##   ..$ colour       : chr "grey20"
##   ..$ size         : NULL
##   ..$ linetype     : NULL
##   ..$ inherit.blank: logi TRUE
##   ..- attr(*, "class")= chr [1:2] "element_rect" "element"
##  $ strip.placement      : chr "inside"
##  $ strip.text           :List of 11
##   ..$ family       : NULL
##   ..$ face         : NULL
##   ..$ colour       : chr "grey10"
##   ..$ size         :Class 'rel'  num 0.8
##   ..$ hjust        : NULL
##   ..$ vjust        : NULL
##   ..$ angle        : NULL
##   ..$ lineheight   : NULL
##   ..$ margin       : NULL
##   ..$ debug        : NULL
##   ..$ inherit.blank: logi TRUE
##   ..- attr(*, "class")= chr [1:2] "element_text" "element"
##  $ strip.text.x         :List of 11
##   ..$ family       : NULL
##   ..$ face         : NULL
##   ..$ colour       : NULL
##   ..$ size         : NULL
##   ..$ hjust        : NULL
##   ..$ vjust        : NULL
##   ..$ angle        : NULL
##   ..$ lineheight   : NULL
##   ..$ margin       :Classes 'margin', 'unit'  atomic [1:4] 5.5 0 5.5 0
##   .. .. ..- attr(*, "valid.unit")= int 8
##   .. .. ..- attr(*, "unit")= chr "pt"
##   ..$ debug        : NULL
##   ..$ inherit.blank: logi TRUE
##   ..- attr(*, "class")= chr [1:2] "element_text" "element"
##  $ strip.text.y         :List of 11
##   ..$ family       : NULL
##   ..$ face         : NULL
##   ..$ colour       : NULL
##   ..$ size         : NULL
##   ..$ hjust        : NULL
##   ..$ vjust        : NULL
##   ..$ angle        : num -90
##   ..$ lineheight   : NULL
##   ..$ margin       :Classes 'margin', 'unit'  atomic [1:4] 0 5.5 0 5.5
##   .. .. ..- attr(*, "valid.unit")= int 8
##   .. .. ..- attr(*, "unit")= chr "pt"
##   ..$ debug        : NULL
##   ..$ inherit.blank: logi TRUE
##   ..- attr(*, "class")= chr [1:2] "element_text" "element"
##  $ strip.switch.pad.grid:Class 'unit'  atomic [1:1] 0.1
##   .. ..- attr(*, "valid.unit")= int 1
##   .. ..- attr(*, "unit")= chr "cm"
##  $ strip.switch.pad.wrap:Class 'unit'  atomic [1:1] 0.1
##   .. ..- attr(*, "valid.unit")= int 1
##   .. ..- attr(*, "unit")= chr "cm"
##  - attr(*, "class")= chr [1:2] "theme" "gg"
##  - attr(*, "complete")= logi TRUE
##  - attr(*, "validate")= logi TRUE

Natural Experiments:

Regression Discontinuity

Two approaches:

• Potential outcomes of \(Y\) are approximately linear near the threshold; treatment shifts outcomes
  • so we (flexibly) model \(Y\) on either side of the cut-off
• Potential outcomes of \(Y\) are random near the cut-off
  • difference in means near the cutoff

Natural Experiments:

Regression Discontinuity

Decisions:

• model outcome around cutoff
  • bias/variance trade-off
  • how do you model it? linear? quadratic? cubic? quartic? local linear?
  • how do you choose the “bandwidth” around the cutoff to use?
  • typically recommend local linear, automatic choice of ‘optimal bandwidth’
• difference in means at cutoff
  • how do you choose the “bandwidth” around the cutoff to use?

Natural Experiments:

Regression Discontinuity

Assumptions:

• assignment to treatment at the threshold is random
• AND
  • either we have correctly modeled potential outcomes of \(Y\) on either side of cutoff
  • or we have used “bandwidth” around cutoff within which treatment is as-if random assigned (difference in means)

Natural Experiments:

Instrumental Variables

Follows from Wald estimator for non-compliance:

• Instrumental Variables Least Squares uses variance “explained” in \(X\) by randomly induced \(Z\)
  • \(X\) is “treatment”, \(Z\) is “random assignment to treatment”
  • easy to understand when \(Z\) is binary
  • when \(Z\) is continuous, it is more complicated (and depends on assumptions of linearity)

Natural Experiments:

Instrumental Variables

Decisions

• is variation induced in \(X\) by \(Z\) linear?
• does \(Z\) actually predict change in \(X\)?
  • “weak instruments problem”: identical to when there are very few compliers in an experiment: inference becomes very hard
  • typically do \(F\) test of regression of \(Y\) on \(X\); \(F\) statistic \(> 10\).

Natural Experiments:

Instrumental Variables

Assumptions:

1. \(Z\) is randomly assigned
2. Exclusion Restriction
   • no other causal path from \(Z\) to \(Y\), other than \(X\)
3. Monotonicity
   • increasing \(Z\) uniformly increases or decreases values of \(X\)

Natural Experiments:

Choices

• Sometimes natural experiments only random conditional on some other variables (Nellis, et al.)
  • What is the correct functional form for the conditioning variables?
• Standard Errors:
  • What is the unit at which treatment is assigned?
  • Cluster/Randomization inference

Natural Experiments:

Limitations

• Effects are “local” or for “compliers”
  • like in non-compliance, we find subset of cases that are randomly induced into treatment/responsive
  • cases may be different
• Causal exposures may be different
  • cause applied at cutoff might be different than elsewhere
  • cause applied at random might be different (e.g. rainfall as instrument for economic growth)

Natural Experiments:

What to do:

• Show robustness to choices of analysis
• Evidence that randomization took place
  • statistical tests of implication
  • qualitative evidence: “causal process observation”